TO-404: remove solution-specific fields - #835

backend/migrations/versions/2026_07_21_1311-8202f7b5953e_replace_solution_specific_fields_with_.py:46

• This migration both introduces the new attributes column and drops the legacy bundled_builds_hash column and artefact_bundled_builds_association table in the same deploy step. In rolling/HA deployments, this can break older application instances still expecting the old schema (expand/contract guideline).

Consider splitting into at least two migrations/PRs: (1) add & backfill attributes while keeping old columns/tables, deploy code that can read/write the new field, then (2) remove the legacy columns/tables in a later deploy.

def _remove_bundled_builds() -> None:
    _copy_bundled_builds_to_attributes()
    op.drop_table("artefact_bundled_builds_association")
    op.drop_column("artefact", "bundled_builds_hash")

backend/migrations/versions/2026_07_21_1311-8202f7b5953e_replace_solution_specific_fields_with_.py:26

• The new unique index on solutions is only (name, version). If the existing database contains multiple solution rows with the same name/version (previously allowed by including source/track/stage/bundled_builds_hash), op.create_index(..., unique=True) will fail during upgrade and block deployment. The migration should either reconcile duplicates (e.g. merge/archive) or proactively detect and raise a clear, actionable error before attempting to create the index.

def upgrade() -> None:
    op.add_column("artefact", sa.Column("attributes", postgresql.JSONB(), server_default="{}", nullable=False))
    op.drop_index("unique_solution", table_name="artefact", postgresql_where="(family = 'solution'::familyname)")
    op.create_index(
        "unique_solution", "artefact", ["name", "version"], unique=True, postgresql_where=sa.text("family = 'solution'")
    )
    _remove_bundled_builds()

backend/test_observer/data_access/models.py:328

• The comment suggests attributes is solution-specific, but the column is defined on the generic Artefact model and is returned by the generic artefact API response. This comment is misleading and makes the field look like a violation of the project’s family-agnostic design, even though the implementation is generic.

    # (for now) Solution specific field

backend/test_observer/controllers/artefacts/models.py:94

• The deprecated bundled_builds field now returns list[Any], which downgrades the OpenAPI schema (items become untyped {}) and is less backwards-compatible for clients that still use the old response shape. Since it is always empty, you can keep the previous element type to preserve the contract while still deprecating it.

    @computed_field(
        deprecated="bundled_builds is deprecated and always empty; solutions now use the generic "
        "attributes field instead.",
    )
    def bundled_builds(self) -> list[Any]:

backend/migrations/versions/2026_07_21_1311-8202f7b5953e_replace_solution_specific_fields_with_.py:42

• This migration both expands (adds artefact.attributes + new unique index) and contracts (drops artefact_bundled_builds_association and artefact.bundled_builds_hash) in the same upgrade. In rolling/HA deployments, applying the migration while old app versions are still running can break them (they may still read/write the dropped table/column). Consider splitting into an expand migration (add attributes + copy data + keep old table/column) and a later contract migration (drop old structures) after the new code is fully deployed.

def upgrade() -> None:
    op.add_column("artefact", sa.Column("attributes", postgresql.JSONB(), server_default="{}", nullable=False))
    _assert_no_duplicate_solutions(["name", "version"])
    op.drop_index("unique_solution", table_name="artefact", postgresql_where="(family = 'solution'::familyname)")
    op.create_index(
        "unique_solution", "artefact", ["name", "version"], unique=True, postgresql_where=sa.text("family = 'solution'")
    )
    _remove_bundled_builds()

backend/test_observer/data_access/models.py:380

• unique_solution is updated in the ORM metadata to only cover (name, version), but the migrations in this PR do not swap the database index accordingly (the new migration is explicitly an expand step that leaves the legacy unique_solution index in place). This schema drift can both break alembic check/autogenerate and leave production DBs upgraded via migrations enforcing a different uniqueness rule than the ORM/runtime code assumes.

Either keep the legacy index/columns represented in Base.metadata until the contract migration is applied, or add the contract migration in this PR (swap unique_solution in the DB, then drop the legacy column/table only after rollout).

        Index(
            "unique_solution",
            "name",
            "version",
            postgresql_where=column("family") == FamilyName.solution.name,

backend/test_observer/controllers/test_executions/start_test.py:353

• For solutions, create_artefact() now calls get_or_create() with filter_kwargs limited to (name, version, family). In a database that already contains multiple solution rows for the same name/version (possible under the current unique_solution definition, and during rollout), get_or_create() will return an arbitrary match (.first()), so new test executions may attach to the wrong artefact. Either enforce the new (name, version) uniqueness at the DB level before using this lookup, or make the lookup deterministic / detect duplicates and fail fast.

            case StartSolutionTestExecutionRequest():
                creation_kwargs["attributes"] = self.request.attributes

        self.artefact = get_or_create(self.db, Artefact, filter_kwargs=filter_kwargs, creation_kwargs=creation_kwargs)

backend/test_observer/data_access/models.py:380

• Artefact model metadata now defines unique_solution as unique on (name, version) only, but the only migration in this PR (2026_07_21_1311-8202f7b5953e_add_artefact_attributes_and_backfill.py) does not recreate unique_solution (and explicitly documents keeping the old schema for rolling upgrades). This mismatch will make alembic check / autogenerate report pending schema changes, and it undermines the “expand”/rolling-upgrade contract described by the migration tests.

        Index(
            "unique_solution",
            "name",
            "version",
            postgresql_where=column("family") == FamilyName.solution.name,

backend/test_observer/controllers/test_executions/models.py:324

• map_legacy_track_and_source mutates the incoming attributes dict (and the data dict) in-place. Because this is a mode="before" validator, the input object may be reused by the caller (e.g., unit tests or internal Python callers), and in-place mutation can lead to surprising side effects. Safer to copy attributes (and return a new data dict) before adding legacy track/source.

            attributes = data["attributes"]
            if attributes is None or not isinstance(attributes, dict):
                return data

backend/migrations/env.py:25

• typing.Tuple is only used for _EXPAND_CONTRACT_IGNORED_COLUMNS. Prefer the builtin tuple (PEP 585) and drop the extra import (this also avoids import-order/isort complaints).

# for 'autogenerate' support
target_metadata = Base.metadata

backend/tests/data_generator.py:168

• attributes = attributes or {} treats an explicitly provided empty dict as "not provided" and replaces it with a new dict. Use an explicit None check so callers passing {} keep their object/value unchanged.

        created_at = created_at or datetime.utcnow()
        reviewers = reviewers or []
        attributes = attributes or {}

backend/migrations/env.py:52

• The include_object callback parameter name object shadows the Python builtin object and may be flagged by Ruff/flake8-builtins. Renaming it also makes it clearer this is an Alembic/SQLAlchemy object.

        return False
    if type_ == "column":
        table_name = object.table.name if object.table is not None else None
        if (table_name, name) in _EXPAND_CONTRACT_IGNORED_COLUMNS:
            return False

backend/test_observer/controllers/artefacts/models.py:75

• Removing the previously-exposed bundled_builds field from the /v1/artefacts/* response is a backward-incompatible API change for existing clients. If clients still depend on this field, consider either (a) keeping it temporarily as a deprecated field (e.g. defaulting to an empty list) or (b) introducing a versioned response (/v2) for the breaking change.

    stage: str
    family: str
    status: ArtefactStatus
    comment: str
    attributes: dict[str, Any]
    archived: bool

backend/test_observer/controllers/artefacts/models.py:151

• Removing the bundled_in field from ArtefactBuildResponse is a backward-incompatible change for clients consuming build details. If this field has active consumers, consider keeping it temporarily as a deprecated field (even if it always returns []) or moving the change behind a versioned API response.

    model_config = ConfigDict(from_attributes=True)

    id: int
    architecture: str
    revision: int | None
    test_executions: list[TestExecutionResponse]